<!DOCTYPE html>
<html>
<head>
    

    

    



    <meta charset="utf-8">
    
    
    <link rel="canonical" href="http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html">
    
    
    <title>Hadoop伪分布式部署 | XieJM&#39;s Blog | 建立博客是为了记录工作经验以及生活点滴,也是将知识和经验分享给需要的朋友，希望对你有帮助！</title>
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
    
    <meta name="theme-color" content="#00bcd4">
    
    
    <meta name="keywords" content="Hadoop">
    <meta name="description" content="本文主要记录Hadoop伪分布式（Pseudo-Distributed Mode）部署的详细步骤：伪分布模式在“单节点集群”上运行Hadoop，其中所有的守护进程都运行在同一台机器上。该模式在单机模式之上增加了代码调试功能，允许你检查内存使用情况，HDFS输入输出，以及其他的守护进程交互。 1.基本环境 CentOS 6.7 JDK1.7+ Hadoop 2.7.4   系统原始环境特别说明：系统">
<meta name="keywords" content="Hadoop">
<meta property="og:type" content="article">
<meta property="og:title" content="Hadoop伪分布式部署">
<meta property="og:url" content="http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html">
<meta property="og:site_name" content="XieJM&#39;s Blog">
<meta property="og:description" content="本文主要记录Hadoop伪分布式（Pseudo-Distributed Mode）部署的详细步骤：伪分布模式在“单节点集群”上运行Hadoop，其中所有的守护进程都运行在同一台机器上。该模式在单机模式之上增加了代码调试功能，允许你检查内存使用情况，HDFS输入输出，以及其他的守护进程交互。 1.基本环境 CentOS 6.7 JDK1.7+ Hadoop 2.7.4   系统原始环境特别说明：系统">
<meta property="og:locale" content="zh-CN">
<meta property="og:updated_time" content="2017-10-03T06:16:06.803Z">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Hadoop伪分布式部署">
<meta name="twitter:description" content="本文主要记录Hadoop伪分布式（Pseudo-Distributed Mode）部署的详细步骤：伪分布模式在“单节点集群”上运行Hadoop，其中所有的守护进程都运行在同一台机器上。该模式在单机模式之上增加了代码调试功能，允许你检查内存使用情况，HDFS输入输出，以及其他的守护进程交互。 1.基本环境 CentOS 6.7 JDK1.7+ Hadoop 2.7.4   系统原始环境特别说明：系统">
    
        <link rel="alternate" type="application/atom+xml" title="XieJM&#39;s Blog" href="/atom.xml">
    
    <link rel="shortcut icon" href="/favicon.ico">
    <link rel="stylesheet" href="/css/style.css?v=1.6.13">
    <script>window.lazyScripts=[]</script>

    <!-- custom head -->
    

</head>

<body>
    <div id="loading" class="active"></div>

    <aside id="menu" class="hide" >
  <div class="inner flex-row-vertical">
    <a href="javascript:;" class="header-icon waves-effect waves-circle waves-light" id="menu-off">
        <i class="icon icon-lg icon-close"></i>
    </a>
    <div class="brand-wrap" style="background-image:url(/img/brand.jpg)">
      <div class="brand">
        <a href="/" class="avatar waves-effect waves-circle waves-light">
          <img src="/img/avatar.jpg">
        </a>
        <hgroup class="introduce">
          <h5 class="nickname">XieJM</h5>
          <a href="mailto:309469843@qq.com" title="309469843@qq.com" class="mail">309469843@qq.com</a>
        </hgroup>
      </div>
    </div>
    <div class="scroll-wrap flex-col">
      <ul class="nav">
        
            <li class="waves-block waves-effect">
              <a href="/index.html"  >
                <i class="icon icon-lg icon-home"></i>
                主页
              </a>
            </li>
        
            <li class="waves-block waves-effect">
              <a href="/archives/index.html"  >
                <i class="icon icon-lg icon-archives"></i>
                归档
              </a>
            </li>
        
            <li class="waves-block waves-effect">
              <a href="/tags/index.html"  >
                <i class="icon icon-lg icon-tags"></i>
                标签
              </a>
            </li>
        
            <li class="waves-block waves-effect">
              <a href="/categories/index.html"  >
                <i class="icon icon-lg icon-th-list"></i>
                分类
              </a>
            </li>
        
            <li class="waves-block waves-effect">
              <a href="https://github.com/xjmhz" target="_blank" >
                <i class="icon icon-lg icon-github"></i>
                Github
              </a>
            </li>
        
      </ul>      
    </div>
    <footer class="footer">
    <p>欢迎加入我们的大数据交流群：<br>群1：258669058 群2：126181630</p>   
    <p>        
        <span><a rel="license" href="https://creativecommons.org/licenses/by-nc-sa/4.0/deed.zh"><img src="/img/cc.png"></a></span>
        
        <span><a href="/atom.xml" target="_blank" class="rss" title="rss"><i class="icon icon-2x icon-rss-square"></i></a></span>
        
    </p>
    <p><span>XieJM &copy; 2017</span>
    </p>
    <p><span>
            
            Power by <a href="http://hexo.io/" target="_blank">Hexo</a> Theme <a href="https://github.com/yscoder/hexo-theme-indigo" target="_blank">indigo</a>
        </span>
    </p>
</footer>
  </div>
</aside>

    <main id="main">
        <header class="top-header" id="header">
    <div class="flex-row">
        <a href="javascript:;" class="header-icon waves-effect waves-circle waves-light on" id="menu-toggle">
          <i class="icon icon-lg icon-navicon"></i>
        </a>
        <div class="flex-col header-title ellipsis">Hadoop伪分布式部署</div>
        
        <div class="search-wrap" id="search-wrap">
            <a href="javascript:;" class="header-icon waves-effect waves-circle waves-light" id="back">
                <i class="icon icon-lg icon-chevron-left"></i>
            </a>
            <input type="text" id="key" class="search-input" autocomplete="off" placeholder="输入感兴趣的关键字">
            <a href="javascript:;" class="header-icon waves-effect waves-circle waves-light" id="search">
                <i class="icon icon-lg icon-search"></i>
            </a>
        </div>
        
        
        <a href="javascript:;" class="header-icon waves-effect waves-circle waves-light" id="menuShare">
            <i class="icon icon-lg icon-share-alt"></i>
        </a>
        
    </div>
</header>
<header class="content-header post-header">

    <div class="container fade-scale">
        <h1 class="title">Hadoop伪分布式部署</h1>
        <h5 class="subtitle">
            
                <time datetime="2017-09-01T15:00:12.343Z" itemprop="datePublished" class="page-time">
  2017-09-01
</time>


	<ul class="article-category-list"><li class="article-category-list-item"><a class="article-category-list-link" href="/categories/tech/">技术</a></li></ul>

            
        </h5>
    </div>

    


</header>


<div class="container body-wrap">
    
    <aside class="post-widget">
        <nav class="post-toc-wrap" id="post-toc">
            <h4>TOC</h4>
            <ol class="post-toc"><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#1-基本环境"><span class="post-toc-text">1.基本环境</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#2-JDK安装"><span class="post-toc-text">2.JDK安装</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#3-创建Hadoop用户"><span class="post-toc-text">3.创建Hadoop用户</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#4-上传（下载）解压Hadoop包"><span class="post-toc-text">4.上传（下载）解压Hadoop包</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#5-配置环境变量"><span class="post-toc-text">5.配置环境变量</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#6-配置SSH"><span class="post-toc-text">6.配置SSH</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#7-配置core-site-xml"><span class="post-toc-text">7.配置core-site.xml</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#8-配置hdfs-site-xml"><span class="post-toc-text">8.配置hdfs-site.xml</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#9-修改mapred-site-xml"><span class="post-toc-text">9.修改mapred-site.xml</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#10-修改yarn-site-xml"><span class="post-toc-text">10.修改yarn-site.xml</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#11-修改slave文件"><span class="post-toc-text">11.修改slave文件</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#12-启动HDFS和YARN"><span class="post-toc-text">12.启动HDFS和YARN</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#13-检查启动进程"><span class="post-toc-text">13.检查启动进程</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#14-验证MapReduce"><span class="post-toc-text">14.验证MapReduce</span></a></li><li class="post-toc-item post-toc-level-2"><a class="post-toc-link" href="#15-参考资料"><span class="post-toc-text">15.参考资料</span></a></li></ol>
        </nav>
    </aside>
    
<article id="post-Hadoop/Hadoop_Pseudo-Distributed_Mode"
  class="post-article article-type-post fade" itemprop="blogPost">

    <div class="post-card">
        <h1 class="post-card-title">Hadoop伪分布式部署</h1>
        <div class="post-meta">
            <time class="post-time" title="2017-09-01 23:00:12" datetime="2017-09-01T15:00:12.343Z"  itemprop="datePublished">2017-09-01</time>

            
	<ul class="article-category-list"><li class="article-category-list-item"><a class="article-category-list-link" href="/categories/tech/">技术</a></li></ul>



            
<span id="busuanzi_container_page_pv" title="文章总阅读量" style='display:none'>
    <i class="icon icon-eye icon-pr"></i><span id="busuanzi_value_page_pv"></span>
</span>


        </div>
        <div class="post-content" id="post-content" itemprop="postContent">
            <p>本文主要记录Hadoop伪分布式（Pseudo-Distributed Mode）部署的详细步骤：<br>伪分布模式在“单节点集群”上运行Hadoop，其中所有的守护进程都运行在同一台机器上。该模式在单机模式之上增加了代码调试功能，允许你检查内存使用情况，HDFS输入输出，以及其他的守护进程交互。</p>
<h2 id="1-基本环境"><a href="#1-基本环境" class="headerlink" title="1.基本环境"></a>1.基本环境</h2><ul>
<li>CentOS 6.7</li>
<li>JDK1.7+</li>
<li>Hadoop 2.7.4</li>
</ul>
<blockquote>
<p>系统原始环境特别说明：<br>系统选择minimal最小系统安装，并自定义选装以下包：<br>Base System中的Base、Compatbillty libraries、Debugging Tools; Development中的 Development tools</p>
</blockquote>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div></pre></td><td class="code"><pre><div class="line"><span class="comment">#关闭防火墙</span></div><div class="line">[root@hadoop001 ~]<span class="comment"># service iptables stop</span></div><div class="line">iptables: Setting chains to policy ACCEPT: filter          [  OK  ]</div><div class="line">iptables: Flushing firewall rules:                         [  OK  ]</div><div class="line">iptables: Unloading modules:                               [  OK  ]</div><div class="line">[root@hadoop001 ~]<span class="comment"># chkconfig iptables off</span></div><div class="line"></div><div class="line"><span class="comment">#关闭SELINUX</span></div><div class="line">[root@hadoop001 ~]<span class="comment"># sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config</span></div><div class="line"><span class="comment">#关闭当前的SELINUX状态,如果不用setenforce那么需要重启系统</span></div><div class="line">[root@hadoop001 ~]<span class="comment"># setenforce 0</span></div><div class="line">[root@hadoop001 ~]<span class="comment"># grep SELINUX=disabled /etc/selinux/config</span></div><div class="line">SELINUX=disabled</div></pre></td></tr></table></figure>
<h2 id="2-JDK安装"><a href="#2-JDK安装" class="headerlink" title="2.JDK安装"></a>2.JDK安装</h2><p>本文采用JDK1.7.79,详细安装步骤参考本博客的JDK安装的文章:<a href="http://xiejm.com/Linux/CentOS_setup_JDK.html">JDK安装</a></p>
<h2 id="3-创建Hadoop用户"><a href="#3-创建Hadoop用户" class="headerlink" title="3.创建Hadoop用户"></a>3.创建Hadoop用户</h2><p>增加ID号为1000的hadoopGroup,增加ID为2000的用户并加入hadoopGroup组</p>
<p>固定用户和组的ID号是为了用户和组的权限一致，我遇到过一次在两台机器上用户名和组名相同，ID号不同结果导致权限有问题</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">[root@hadoop001 ~]<span class="comment"># groupadd -g 1000 hadoopGroup</span></div><div class="line">[root@hadoop001 ~]<span class="comment"># useradd -u 2000 -g hadoopGroup hadoop</span></div></pre></td></tr></table></figure>
<p>设置hadoop用户的密码为hadoop，下面的这种方法免去了修改密码时的密码确认</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">[root@hadoop001 ~]<span class="comment"># echo "hadoop"|passwd --stdin hadoop</span></div></pre></td></tr></table></figure>
<p>设置hadoop用户sodu权限</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">[root@hadoop001 ~]<span class="comment"># vi /etc/sudoers</span></div><div class="line">hadoop ALL=(ALL)       ALL</div></pre></td></tr></table></figure>
<h2 id="4-上传（下载）解压Hadoop包"><a href="#4-上传（下载）解压Hadoop包" class="headerlink" title="4.上传（下载）解压Hadoop包"></a>4.上传（下载）解压Hadoop包</h2><p>hadoop-2.7.4.tar.gz包使用rz命令上传或通过wget命令去官网下载</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div></pre></td><td class="code"><pre><div class="line">[root@hadoop001 ~]<span class="comment"># mkdir -p /opt/software</span></div><div class="line"><span class="comment">#将hadoop安装包放到/opt/software</span></div><div class="line">[root@hadoop001 ~]<span class="comment"># cd /opt/software</span></div><div class="line">[root@hadoop001 software]<span class="comment"># tar -zxvf hadoop-2.7.4.tar.gz</span></div><div class="line"></div><div class="line"><span class="comment">#设置软连接</span></div><div class="line">[root@hadoop001 software]<span class="comment"># ln -s /opt/software/hadoop-2.7.4 /opt/software/hadoop</span></div><div class="line"></div><div class="line"><span class="comment">#设置hadoop目录所有者、所属组</span></div><div class="line">[root@hadoop001 software]<span class="comment"># chown -R hadoop:hadoopGroup hadoop/*</span></div><div class="line">[root@hadoop001 software]<span class="comment"># chown -R hadoop:hadoopGroup hadoop-2.7.4</span></div><div class="line">[root@hadoop001 software]<span class="comment"># chown -R hadoop:hadoopGroup hadoop-2.7.4/*</span></div></pre></td></tr></table></figure>
<h2 id="5-配置环境变量"><a href="#5-配置环境变量" class="headerlink" title="5.配置环境变量"></a>5.配置环境变量</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div></pre></td><td class="code"><pre><div class="line">[root@hadoop001 software]<span class="comment"># vim /etc/profile</span></div><div class="line"></div><div class="line"><span class="comment">#按组合键shift+g 或大写G 到最后一行，按字母o  插入下列内容</span></div><div class="line">    <span class="comment">#SET HADOOP</span></div><div class="line">    exprot HADOOP_HOME=/opt/software/hadoop</div><div class="line"><span class="comment">#如果JDK环境变量没有配置，请加入下列内容</span></div><div class="line">    <span class="comment">#SET JDK</span></div><div class="line">    <span class="built_in">export</span> JAVA_HOME=/usr/java/jdk1.7.0_79</div><div class="line">    <span class="built_in">export</span> JRE_HOME=<span class="variable">$JAVA_HOME</span>/jre</div><div class="line">    <span class="built_in">export</span> CLASS_PATH=.:<span class="variable">$JAVA_HOME</span>/lib:<span class="variable">$JAVA_HOME</span>/lib/tools.jar</div><div class="line"></div><div class="line"><span class="comment">#设置PATH</span></div><div class="line">    <span class="built_in">export</span> PATH=<span class="variable">$HADOOP_HOME</span>/bin:<span class="variable">$HADOOP_HOME</span>/sbin:<span class="variable">$&#123;JAVA_HOME&#125;</span>/bin:<span class="variable">$&#123;JRE_HOME&#125;</span>/bin:<span class="variable">$PATH</span></div><div class="line"></div><div class="line"><span class="comment">#添加完成后  :wq 保存</span></div><div class="line"><span class="comment">#使修改生效：</span></div><div class="line">[root@hadoop001 software]<span class="comment">#source !$</span></div><div class="line"></div><div class="line"><span class="comment">#验证JDK有效性</span></div><div class="line">[root@hadoop001 software]<span class="comment"># java -version</span></div><div class="line">java version <span class="string">"1.7.0_79"</span></div><div class="line">Java(TM) SE Runtime Environment (build 1.7.0_79-b15)</div><div class="line">Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)</div></pre></td></tr></table></figure>
<h2 id="6-配置SSH"><a href="#6-配置SSH" class="headerlink" title="6.配置SSH"></a>6.配置SSH</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div></pre></td><td class="code"><pre><div class="line"><span class="comment">#检查sshd</span></div><div class="line">[root@hadoop001 software]<span class="comment"># service sshd status</span></div><div class="line">openssh-daemon (pid  1844) is running...</div><div class="line">[root@hadoop001 software]<span class="comment"># su - hadoop</span></div><div class="line"></div><div class="line">[hadoop@hadoop001 ~]<span class="comment"># cd /opt/software/hadoop</span></div><div class="line"></div><div class="line"><span class="comment">#SSH 免密设置</span></div><div class="line"><span class="comment">#由于是伪分布式需要用到ssh localhost的情况，所以需要在本机生成公钥</span></div><div class="line">[hadoop@hadoop001 hadoop]$ ssh-keygen -t rsa -P <span class="string">''</span> -f ~/.ssh/id_rsa</div><div class="line">[hadoop@hadoop001 hadoop]$ cat ~/.ssh/id_rsa.pub &gt;&gt; ~/.ssh/authorized_keys</div><div class="line">[hadoop@hadoop001 hadoop]$ chmod 0600 ~/.ssh/authorized_keys</div><div class="line"></div><div class="line"><span class="comment"># 验证SSH</span></div><div class="line">[hadoop@hadoop001 hadoop]<span class="variable">$ssh</span> hadoop001 date</div><div class="line">The authenticity of host <span class="string">'hadoop001 (192.168.137.130)'</span> can<span class="string">'t be established.</span></div><div class="line"><span class="string">RSA key fingerprint is 09:f6:4a:f1:a0:bd:79:fd:34:e7:75:94:0b:3c:83:5a.</span></div><div class="line"><span class="string">Are you sure you want to continue connecting (yes/no)? yes   #第一次回车输入yes</span></div><div class="line"><span class="string"></span></div><div class="line"><span class="string">Warning: Permanently added '</span>hadoop001,192.168.137.130<span class="string">' (RSA) to the list of known hosts.</span></div><div class="line"><span class="string">Sun Aug 20 14:22:28 CST 2017</span></div><div class="line"><span class="string"></span></div><div class="line"><span class="string">[hadoop@hadoop001 hadoop]$ssh hadoop001 date   #不需要回车输入yes,即SSH配置成功</span></div><div class="line"><span class="string">Sun Aug 20 14:22:29 CST 2017</span></div></pre></td></tr></table></figure>
<h2 id="7-配置core-site-xml"><a href="#7-配置core-site-xml" class="headerlink" title="7.配置core-site.xml"></a>7.配置core-site.xml</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">[hadoop@hadoop001 hadoop]$ vi etc/hadoop/core-site.xml</div><div class="line"><span class="comment">#配置HDFS地址</span></div><div class="line">&lt;configuration&gt;</div><div class="line">    &lt;property&gt;</div><div class="line">        &lt;name&gt;fs.defaultFS&lt;/name&gt;</div><div class="line">        &lt;value&gt;hdfs://hadoop001:9000&lt;/value&gt;</div><div class="line">    &lt;/property&gt;</div><div class="line">&lt;/configuration&gt;</div></pre></td></tr></table></figure>
<h2 id="8-配置hdfs-site-xml"><a href="#8-配置hdfs-site-xml" class="headerlink" title="8.配置hdfs-site.xml"></a>8.配置hdfs-site.xml</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div></pre></td><td class="code"><pre><div class="line">[hadoop@hadoop001 hadoop]$ vi etc/hadoop/hdfs-site.xml</div><div class="line"><span class="comment">#配置HDFS副本数</span></div><div class="line">&lt;configuration&gt;</div><div class="line">    &lt;property&gt;</div><div class="line">        &lt;name&gt;dfs.replication&lt;/name&gt;</div><div class="line">        &lt;value&gt;1&lt;/value&gt;</div><div class="line">    &lt;/property&gt;</div><div class="line">    &lt;property&gt;</div><div class="line">        &lt;name&gt;dfs.namenode.secondary.http-address&lt;/name&gt;</div><div class="line">        &lt;value&gt;hadoop001:50090&lt;/value&gt;</div><div class="line">    &lt;/property&gt;</div><div class="line">    &lt;property&gt;</div><div class="line">        &lt;name&gt;dfs.namenode.secondary.https-address&lt;/name&gt;</div><div class="line">        &lt;value&gt;hadoop001:50091&lt;/value&gt;</div><div class="line">    &lt;/property&gt;</div><div class="line">&lt;/configuration&gt;</div></pre></td></tr></table></figure>
<h2 id="9-修改mapred-site-xml"><a href="#9-修改mapred-site-xml" class="headerlink" title="9.修改mapred-site.xml"></a>9.修改mapred-site.xml</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">[hadoop@hadoop001 hadoop]$ vi etc/hadoop/mapred-site.xml:</div><div class="line"><span class="comment">#设置mapreduce框架为yarn</span></div><div class="line">&lt;configuration&gt;</div><div class="line">    &lt;property&gt;</div><div class="line">        &lt;name&gt;mapreduce.framework.name&lt;/name&gt;</div><div class="line">        &lt;value&gt;yarn&lt;/value&gt;</div><div class="line">    &lt;/property&gt;</div><div class="line">&lt;/configuration&gt;</div></pre></td></tr></table></figure>
<h2 id="10-修改yarn-site-xml"><a href="#10-修改yarn-site-xml" class="headerlink" title="10.修改yarn-site.xml"></a>10.修改yarn-site.xml</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">[hadoop@hadoop001 hadoop]$ etc/hadoop/yarn-site.xml:</div><div class="line"></div><div class="line">&lt;configuration&gt;</div><div class="line">    &lt;property&gt;</div><div class="line">        &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;</div><div class="line">        &lt;value&gt;mapreduce_shuffle&lt;/value&gt;</div><div class="line">    &lt;/property&gt;</div><div class="line">&lt;/configuration&gt;</div></pre></td></tr></table></figure>
<h2 id="11-修改slave文件"><a href="#11-修改slave文件" class="headerlink" title="11.修改slave文件"></a>11.修改slave文件</h2><p>将本机的主机名写入slave节点</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">ehco <span class="string">'hadoop001'</span> &gt;/opt/software/hadoop/etc/hadooop/slave</div></pre></td></tr></table></figure>
<h2 id="12-启动HDFS和YARN"><a href="#12-启动HDFS和YARN" class="headerlink" title="12.启动HDFS和YARN"></a>12.启动HDFS和YARN</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line"><span class="comment">#添加环境变量</span></div><div class="line">[hadoop@hadoop001 hadoop]$  vi etc/hadoop/hadoop-env.sh</div><div class="line"><span class="built_in">export</span> JAVA_HOME=/usr/java/jdk1.7.0_79</div><div class="line"></div><div class="line">[hadoop@hadoop001 hadoop]$ <span class="built_in">which</span> hdfs</div><div class="line">/opt/software/hadoop/bin/hdfs</div><div class="line">[hadoop@hadoop001 hadoop]$ hdfs namenode -format</div><div class="line">........</div><div class="line">17/08/20 16:43:19 INFO common.Storage: Storage directory</div><div class="line">/tmp/hadoop-hadoop/dfs/name has been successfully formatted.</div><div class="line"></div><div class="line">[hadoop@hadoop001 hadoop]$ <span class="built_in">which</span> start-dfs.sh</div><div class="line">/opt/software/hadoop/sbin/start-dfs.sh</div><div class="line">[hadoop@hadoop001 hadoop]$</div><div class="line">[hadoop@hadoop001 hadoop]$ start-dfs.sh</div><div class="line">[hadoop@hadoop001 hadoop]$ sbin/start-yarn.sh</div><div class="line">[hadoop@hadoop001 hadoop]$ sbin/mr-jobhistory-daemon.sh start historyserver</div></pre></td></tr></table></figure>
<p>如果要结束进程，使用如下命令</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">[hadoop@hadoop001 hadoop]$ sbin/stop-yarn.sh</div><div class="line">[hadoop@hadoop001 hadoop]$ stop-dfs.sh</div><div class="line">[hadoop@hadoop001 hadoop]$ sbin/mr-jobhistory-daemon.sh stop historyserver</div></pre></td></tr></table></figure>
<h2 id="13-检查启动进程"><a href="#13-检查启动进程" class="headerlink" title="13.检查启动进程"></a>13.检查启动进程</h2><p>通过jps命令查看</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">[hadoop@hadoop001 hadoop]$ jps</div><div class="line">19536 DataNode</div><div class="line">19440 NameNode</div><div class="line">19876 Jps</div><div class="line">19740 SecondaryNameNode</div><div class="line">19888 ResourceManager</div><div class="line">19345 NodeManager</div><div class="line">4613 JobHistoryServer</div></pre></td></tr></table></figure>
<p>通过WebUI查看</p>
<blockquote>
<p>Browse the web interface for the NameNode - <a href="http://hadoop001:50070/" target="_blank" rel="external"></a><br>Browse the web interface for the ResourceManager - <a href="http://hadoop001:8088/" target="_blank" rel="external"></a></p>
</blockquote>
<h2 id="14-验证MapReduce"><a href="#14-验证MapReduce" class="headerlink" title="14.验证MapReduce"></a>14.验证MapReduce</h2><p>我们使用hadoop 的mapreduce-examples  jar包来计算PI</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line"><span class="comment">#Run a MapReduce job.</span></div><div class="line">[hadoop@hadoop001 mapreduce]$ <span class="built_in">pwd</span></div><div class="line">/opt/software/hadoop/share/hadoop/mapreduce</div><div class="line">[hadoop@hadoop002 mapreduce]$</div><div class="line"></div><div class="line">[hadoop@hadoop002 mapreduce]$ ll</div><div class="line">-rw-r--r--. 1 hadoop hadoop  301740 Aug 20 15:12 hadoop-mapreduce-examples-2.7.4.jar</div><div class="line">[hadoop@hadoop002 mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.7.4.jar pi 5 10</div></pre></td></tr></table></figure>
<h2 id="15-参考资料"><a href="#15-参考资料" class="headerlink" title="15.参考资料"></a>15.参考资料</h2><p> <a href="http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html" target="_blank" rel="external">Setting up a Single Node Cluster：</a></p>

        </div>

        <blockquote class="post-copyright">
    <div class="content">
        
<span class="post-time">
    最后更新时间：<time datetime="2017-10-03T06:16:06.803Z" itemprop="dateUpdated">2017-10-03 14:16:06</time>
</span><br>


        
        原始链接：<a href="/Hadoop/Hadoop_Pseudo-Distributed_Mode.html" target="_blank" rel="external">http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html</a>
        
    </div>
    <footer>
        <a href="http://xiejm.com">
            <img src="/img/avatar.jpg" alt="XieJM">
            XieJM
        </a>
    </footer>
</blockquote>

        


        <div class="post-footer">
            
	<ul class="article-tag-list"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Hadoop/">Hadoop</a></li></ul>


            
<div class="page-share-wrap">
    

<div class="page-share" id="pageShare">
    <ul class="reset share-icons">
      <li>
        <a class="weibo share-sns" target="_blank" href="http://service.weibo.com/share/share.php?url=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html&title=《Hadoop伪分布式部署》 — XieJM's Blog&pic=http://xiejm.com/img/avatar.jpg" data-title="微博">
          <i class="icon icon-weibo"></i>
        </a>
      </li>
      <li>
        <a class="weixin share-sns wxFab" href="javascript:;" data-title="微信">
          <i class="icon icon-weixin"></i>
        </a>
      </li>
      <li>
        <a class="qq share-sns" target="_blank" href="http://connect.qq.com/widget/shareqq/index.html?url=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html&title=《Hadoop伪分布式部署》 — XieJM's Blog&source=Linux | Hadoop | CDH | Hive | Hbase | Spark" data-title=" QQ">
          <i class="icon icon-qq"></i>
        </a>
      </li>
      <li>
        <a class="facebook share-sns" target="_blank" href="https://www.facebook.com/sharer/sharer.php?u=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html" data-title=" Facebook">
          <i class="icon icon-facebook"></i>
        </a>
      </li>
      <li>
        <a class="twitter share-sns" target="_blank" href="https://twitter.com/intent/tweet?text=《Hadoop伪分布式部署》 — XieJM's Blog&url=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html&via=http://xiejm.com" data-title=" Twitter">
          <i class="icon icon-twitter"></i>
        </a>
      </li>
      <li>
        <a class="google share-sns" target="_blank" href="https://plus.google.com/share?url=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html" data-title=" Google+">
          <i class="icon icon-google-plus"></i>
        </a>
      </li>
    </ul>
 </div>



    <a href="javascript:;" id="shareFab" class="page-share-fab waves-effect waves-circle">
        <i class="icon icon-share-alt icon-lg"></i>
    </a>
</div>



        </div>
    </div>

    
<nav class="post-nav flex-row flex-justify-between">
  
    <div class="waves-block waves-effect prev">
      <a href="/Hadoop/Hadoop--HDFSdesign.html" id="post-prev" class="post-nav-link">
        <div class="tips"><i class="icon icon-angle-left icon-lg icon-pr"></i> Prev</div>
        <h4 class="title">Hadoop--HDFS架构</h4>
      </a>
    </div>
  

  
    <div class="waves-block waves-effect next">
      <a href="/Hadoop/Hadoop_source_code_building.html" id="post-next" class="post-nav-link">
        <div class="tips">Next <i class="icon icon-angle-right icon-lg icon-pl"></i></div>
        <h4 class="title">Hadoop 源码编译安装</h4>
      </a>
    </div>
  
</nav>



    


<section class="comments" id="comments">
    <div id="disqus_thread"></div>
    <script>
    var disqus_shortname = 'true';
    lazyScripts.push('//' + disqus_shortname + '.disqus.com/embed.js')
    </script>
    <noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>
</section>













</article>



</div>

    </main>
    <div class="mask" id="mask"></div>
<a href="javascript:;" id="gotop" class="waves-effect waves-circle waves-light"><span class="icon icon-lg icon-chevron-up"></span></a>



<div class="global-share" id="globalShare">
    <ul class="reset share-icons">
      <li>
        <a class="weibo share-sns" target="_blank" href="http://service.weibo.com/share/share.php?url=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html&title=《Hadoop伪分布式部署》 — XieJM's Blog&pic=http://xiejm.com/img/avatar.jpg" data-title="微博">
          <i class="icon icon-weibo"></i>
        </a>
      </li>
      <li>
        <a class="weixin share-sns wxFab" href="javascript:;" data-title="微信">
          <i class="icon icon-weixin"></i>
        </a>
      </li>
      <li>
        <a class="qq share-sns" target="_blank" href="http://connect.qq.com/widget/shareqq/index.html?url=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html&title=《Hadoop伪分布式部署》 — XieJM's Blog&source=Linux | Hadoop | CDH | Hive | Hbase | Spark" data-title=" QQ">
          <i class="icon icon-qq"></i>
        </a>
      </li>
      <li>
        <a class="facebook share-sns" target="_blank" href="https://www.facebook.com/sharer/sharer.php?u=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html" data-title=" Facebook">
          <i class="icon icon-facebook"></i>
        </a>
      </li>
      <li>
        <a class="twitter share-sns" target="_blank" href="https://twitter.com/intent/tweet?text=《Hadoop伪分布式部署》 — XieJM's Blog&url=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html&via=http://xiejm.com" data-title=" Twitter">
          <i class="icon icon-twitter"></i>
        </a>
      </li>
      <li>
        <a class="google share-sns" target="_blank" href="https://plus.google.com/share?url=http://xiejm.com/Hadoop/Hadoop_Pseudo-Distributed_Mode.html" data-title=" Google+">
          <i class="icon icon-google-plus"></i>
        </a>
      </li>
    </ul>
 </div>


<div class="page-modal wx-share" id="wxShare">
    <a class="close" href="javascript:;"><i class="icon icon-close"></i></a>
    <p>扫一扫，分享到微信</p>
    <img src="" alt="微信分享二维码">
</div>




    <script src="//cdn.bootcss.com/node-waves/0.7.4/waves.min.js"></script>
<script>
var BLOG = { ROOT: '/', SHARE: true, REWARD: false };


</script>

<script src="/js/main.min.js?v=1.6.13"></script>


<div class="search-panel" id="search-panel">
    <ul class="search-result" id="search-result"></ul>
</div>
<template id="search-tpl">
<li class="item">
    <a href="{path}" class="waves-block waves-effect">
        <div class="title ellipsis" title="{title}">{title}</div>
        <div class="flex-row flex-middle">
            <div class="tags ellipsis">
                {tags}
            </div>
            <time class="flex-col time">{date}</time>
        </div>
    </a>
</li>
</template>

<script src="/js/search.min.js?v=1.6.13" async></script>






<script async src="//dn-lbstatics.qbox.me/busuanzi/2.3/busuanzi.pure.mini.js"></script>



<script>
(function() {
    var OriginTitile = document.title, titleTime;
    document.addEventListener('visibilitychange', function() {
        if (document.hidden) {
            document.title = 'XieJM's Blog';
            clearTimeout(titleTime);
        } else {
            document.title = 'XieJM's Blog';
            titleTime = setTimeout(function() {
                document.title = OriginTitile;
            },2000);
        }
    });
})();
</script>



</body>
</html>
